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Abstract 



In synchronous rewriting, the productions 
of two rewriting systems are paired and 
apphed synchronously in the derivation of 
a pair of strings. We present a new syn- 
chronous rewriting system and argue that 
it can handle certain phenomena that are 
not covered by existing synchronous sys- 
tems. We also prove some interesting for- 
mal/computational properties of our sys- 
tem. 

1 Introduction 

Much of theoretical linguistics can be formulated in 



niemi, 1983; Kaplan and Kay, 1994), and more re- 
cently in parsing as well (see, e.g., ( Gross, 198S| ; 



pereira, 1991 ; Roche, 1993| )). Pushdown transduc- 
ers and syntax directed translation schemata (SDTS) 
(Aho and UUman, 1969) translate between context- 
free languages and are therefore more powerful than 
finite state transducers. Pushdown transducers are 
a standard model for parsing, and have also been 
used (usually implicitly) in speech understanding. 
Recently, variants of SDTS have been proposed as 
model s for simu ltaneously bracketing parallel cor- 
pora ( Wu, 19"95| ). Syn chronization of tree adjoin- 



ing grammars (TAGs) (IShieber and Schabes, 19"9C| : 
IShieber, 199^ ) arc even more powerful than the pre- 
vious formalisms, and have been applied in machine 
translation ([Abeillc, Schabes, and Joshi, 199C ; Eged: 



a very natural manner as stating correspondences and Palmer, 1994 [Harbusch and Poller, 1994; |Pri- 



example, related interlace layers LI'' and PI'' in Glj and Schabes, 199l|), and theoretical syntax (Abeille 



and Minimalism (Uhomsky, I'd'd'i), semantic and 



syntactic information m HPSG ( Pollard and Sag^ 
1994), or the different structures such as c-structure 



(translations) between layers ot representation; for gent, 1994]), natur al language generation (^hicber 



and f-structure in LFG (Bresnan and Kaplan, 1982). 
Similarly, many problems in natural language pro- 
cessing, in particular parsing and generation, can be 
expressed as transductions, which are calculations 
of such correspondences. There is therefore a great 
need for formal models of corresponding levels of 
representation, and for corresponding algorithms for 
transduction. 

Several different transduction systems have been 
used in the past by the computational and theoret- 



1994). The common underlying idea in all of these 
formalisms is to combine two generative devices 
through a pairing of their productions (or, in the 
case of the corresponding automata, of their tran- 
sitions) in such a way that right-hand side nonter- 
minal symbols in the paired productions are linked. 
The processes of derivation proceed synchronously 
in the two devices by applying the paired grammar 
rules only to linked nonterminals introduced previ- 
ously in the derivation. The fact that the above sys- 
tems all reflect the same translation technique has 
not always been recognized in the computational lin- 



guistics literature. Following (Shieber and Schabes 



been borrowed trom translation ttieory, a subheid 
of formal language theory, or have been originally 
(and sometimes redundantly) developed. Finite 
state transducers ( for an overview, see, e.g., 

(M 

and UUman, 1972| )) provide translations between 
regular languages. These devices have been pop- 
ular in computational morphology and computa- 
tional phonology since the early eighties ( Kosken- 



Lcal linguistics communities. IJiese systems tiave i99o|) ^e will refer to the general approach as syn 



chronous rewriting. While synchronous systems are 
becoming more and more popular, surprisingly little 
is known about the formal characteristics of these 
systems (with the exception of the finite-state de- 
vices) . 

In this paper, we argue that existing synchronous 
systems cannot handle, in a computationally attrac- 



tive way, a standard problem in syntax/semantics 
translation, namely quantifier scoping. We propose 
a new system that provides a synchronization be- 
tween two unordered vector grammars with domi- 
nance hnks (UVG-DL) ( |Rambow, 1994| ). The type 
of synchronization is closely based on a previously 
proposed model, which we will call "local" synchro- 
nization. We argue that this synchronous system can 
deal with quantifier scoping in the desired way. The 
proposed system has the weak language preservation 
property, that is, the defined synchronization mech- 
anism does not alter the weak generative capacity 
of the formalism being synchronized. Furthermore, 
the tree-to-forest translation problem for our system 
can be solved in polynomial time; that is, given a 
derivation tree obtained according to one of the syn- 
chronized grammars, we can construct the forest of 
all the translated derivation trees in the other gram- 
mar, using a polynomial amount of time. 

The structure of this paper is as follows. In Sec- 
tion ^, we introduce quantifier raising and review 
two types of synchronization and mention some new 
formal results. We introduce our new synchronous 
system in Section ^ and present our formal results 
and outline the proof techniques in Section ^. 

2 Types of Synchronization 
2.1 Quantifier Raising 

We start by presenting an example which is based 
on transfer between a syntactic representation and 
a "semantic" representation of the scoping of quan- 
tified NPs. It is generally assumed that in English 
(and many other languages), quantified arguments 
of a verb can (in appropriate contexts) take scope 
in any possible order, and that this generalization 
extends to cases of embedded clauses ( May, 1985 
For example, sentence (0) can have four possible in- 
terpretations (of the six possible orderings of the 
quantifiers, two pairs are logically equivalent), two 
of which are shown in (^). 

(1) Every man thinks some official said some Nor- 
wegian arrived 



(2) a. Va;, x a man, Ely, y an official, 3z, z a Nor- 
wegian, X thinks y said z arrived 

D. ziz , z a Norwegian, 3y, y an official, Vx, x a 
man, x thinks y said z arrived 



^We explicitly exclude from our analysis cases of 
quantified NPs embedded in NPs, and do not, of course, 
propose to develop a serious linguistic tlieory of quanti- 
fier scoping. 



We give a simplified syntactic representation for 
(|l|) in Figure |], and a simplified semantic represen- 
tation for in Figure ^. 




; official VP 



said 



some Norwegian arrived 

Figure 1: Syntactic representation for (1) 
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z a Norwegian 



exists y, 
y an official 



for all X, 
X a man 
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Figure 2: Semantic representation for (2b) 

2.2 Non- Local Synchronization 

We will first discuss a type of synchronization pro- 



posed by ( Shieber and Schabes, 1990| ), based on 
TAG. We will refer to this system as non-local syn- 
chronous TAG (nlSynchTAG) . The synchronization 
is non-local in the sense that once links are intro- 
duced during a derivation by a synchronized pair of 
grammar rules, they need not continue to impinge on 
the nodes that introduced them: the links may be re- 
assigned to a newly introduced nonterminal when an 
original node is rewritten. We will refer to this mech- 
anism as link inheritance. To illustrate, we will give 
as an example an analysis of the quantifier-raising 
example introduced above, extending in a natural 
manner an example given by Shieber and Schabes. 

The elementary structures are shown in Figure ^ 
(we only give one NP — the others are similar) . The 
nominal arguments in the syntax are associated with 



for all X, 
X a man 




Figure 3: Elementary structures in nlSynchTAG 



pairs of trees in the semantics, and are linked to two 
nodes, the quantifier and the variable. The deriva- 
tion proceeds as illustrated in Figure ^ finally yield- 
ing the two structures in Figure |^ and Figure |^. Note 
that some of the links originating with the NP nodes 
are inherited during the derivation. By changing the 
order in which we add the nominal arguments at the 
end of the derivation, we can obtain all quantifier 
scopes in the semantics. 

The problem with non-local synchronization is 
that the weak language preservation property does 
not hold. (Shieber, 1994) shows that not all 
nlSynchTAG left-projection languages can be gen- 
erated by T AGs. As a new result, in (Rambow and 
Satta, I99t ) we show that the recognition of some 
fixed left-projection languages of a nlSynchTAG is 
NP-complete. Our reduction crucially relies on link 
inheritance. This makes nlSynchTAG unattractive 
for applications in theoretical or computational lin- 
guistics. 



2.3 Local Synchronous Systems 

In contrast with non-local synchronization, in local 
synchronization there is no inheritance of synchro- 
nization links. This is enforced by requiring that 
the links establish a bijection between nonterminals 
in the two synchronously derived sentential forms, 
that is, each nonterminal must be involved in exactly 
one link. In this way, once a nonterminal is rewrit- 
ten through the application of a pair of rules to two 




Figure 4: Non-local derivation in nlSynchTAG 



linked nonterminals, no additional link remains to 
be transferred to the newly introduced nonterminals. 
As a consequence of this, the derivation structures in 
the left and right grammars are always isomorphic 
(up to ordering and labeling of nodes). 

The ca nonical example of local synchronization 
is SDTS (|Aho and Ullman, 1969|) , in which two 
context-free grammars are synchronized. We give 
an example of an SDTS and a derivation in Fig- 
ure H The links are indicated as boxed numbers 
to the right of the nonterminal to which they ap- 
ply. ( jShieber, 1994 ) defines the tree-rewriting ver- 
sion of SDTS, which we will call synchronous TAG 
(SynchTAG), and argues that Synch TAG does not 
have the formal problems of nlSynchTAG (though 



Grammar: 

S |T] NP|T| likes NP|1] S |T] NP|1] piail a NP|1] 



NP 
NP 



Jo/in NP 
the white NP 



Jean 

la N|~6~| blanche 



N| 7 I ^ house 
Derivation: 

(S0, S0) 

=i>(NP|T| Elites NP|1], NP|T| p/ajt a NP|T|) 
=>(NP[T] likes the white N|~6~|, Nfe^ blanche plait a 
NP|T|) 

{John likes the white house, la maison blanche 
plait a Jean) 

Figure 5: Sample SDTS and derivation 



(Shicbcr, 1994) studies the translation problem mak- 
ing the unappealing assumption that each tree in the 
input grammar is associated with only one output 
grammar tree). 

However, Synch TAG cannot derive all possible 
scope orderings, because of the locality restriction. 
This can be shown by adapting the proof technique 



in (Becker, Rambow, and Niv, 1992). In the follow- 
ing section, we will present a synchronous system 
which has local synchronization's formal advantages, 
but handles the scoping data. 

3 Extended Local Synchronization 

In this section, we propose a new synchronous sys- 
tem, which is based on local synchronization of 
unordered vector grammars with dominance links 
(UVG-DL) dRambow, l"99^). The presentations will 



be informal for reasons of space; we refer to (Ram- 



bow and Satta, 1996) for details. In UVG-DL, sev- 



eral context-free string rewriting rules are grouped 
into sets, called vectors. In a derivation, all or no 
rules from a given instance of a vector must be used. 
Put differently, all productions from a given vector 
must be used the same number of times. They can 
be applied in any order and need not be applied 
simultaneously or one right after the other. In addi- 
tion, UVG-DL has dominance links. An occurrence 
of a nonterminal A in the right-hand side of a rule p 
can be linked to the left-hand nonterminal of another 
rule p' in the same vector. This dominance link will 
act as a constraint on derivations: if p is used in 
a derivation, then p' must be used subsequently in 
the subderivation that starts with the occurrence of 
A introduced by p. A UVG-DL is lexicalized iff at 
least one production in every vector contains a ter- 
minal symbol. Henceforth, all UVG-DLs mentioned 
in this paper will implicitly be assumed to be lex- 
icalized. The derivation structure of a UVG-DL is 
just the derivation structure of the same derivation 



in the underlying context-free grammar (the CFG 
obtained by forming the union of all vectors). We 
give an example of a UVG-DL in Figure |6[ in which 
the dotted lines represent the dominance links. A 
sample derivation is in Figure 0. 



{ 
{ 
{ 



for all X, F 
X a man 



exists y, F 
y an official 



} {A} 



} {A} 

say T F 

1 




exists z, F 
z a Norwegian \ 



Figure 6: A UVG-DL for deriving semantic repre- 
sentations such as (2) 

Our proposal for the synchronization of two UVG- 
DL uses the notion of locality in synchronization, 
but with respect to entire vectors, not individual 
productions in these vectors. This approach, as we 
will see, gives us both the desired empirical coverage 
and acceptable computational and formal results. 
We suppose that in each vector u of a UVG-DL there 
is exactly one privileged element, which we call the 
synchronous production of v. All other elements of 
V are referred to as asynchronous productions. In 
Figures ^ and ^, the synchronous productions are 
designated by a bold-italic left-hand side symbol. 
Furthermore, in the right-hand side of each asyn- 
chronous production of v we identify a single non- 
terminal, called the heir. 

In a synchronous UVG-DL (SynchUVG-DL), vec- 
tors from one UVG-DL are synchronized with vec- 
tors from another UVG-DL. Two vectors are syn- 
chronized by specifying a bijective synchronization 
mapping (as in local synchronization) between the 
non-heir right-hand side occurrences of nonterminals 
in the productions of the two vectors. A nontermi- 
nal on which a synchronization link impinges is re- 
ferred to as a synchronous nonterminal. A sample 
SynchUVG-DL grammar is shown in Figure ^. 

Informally speaking, during a SynchUVG-DL 
derivation, the two synchronous productions in a 
pair of synchronized vectors must be applied at 
the same time and must rewrite linked occurrences 
of nonterminals previously introduced. The asyn- 
chronous productions of the two synchronized gram- 



mars are not subject to the synchronization require- 
ment, and they can be apphed at any time and in- 
dependently of the other grammar (but of course 
subject to the grammar-specific dominance hnks). 
Any synchronous hnks that impinge on a nonter- 
minal rewritten by an asynchronous production are 
transferred to the heir of the asynchronous produc- 
tion. (A production may introduce a synchronous 
nonterminal whose counterpart in the other gram- 
mar has not yet been introduced. In this case, the 
link remains "pending" .) Thus, while in SynchUVG- 
DL there is link inheritance as in non-local synchro- 
nization, link inheritance is only possible with those 
productions that themselves are not subject to the 
synchronization requirement. 

The locality of synchronization in SynchUVG- 
DL becomes clear when we consider a new struc- 
ture which we introduce here, called the vec- 
tor derivation tree. Consider two synchronized 
UVG-DLderivations in a SynchUVG-DL. The vector 
derivation tree for either component derivation is ob- 
tained as follows. Each instance of a vector used in 
the derivation is represented as a single node (which 
we label with that vector's lexeme). A node rep- 
resenting a vector vi is immediately dominated by 
the node representing the vector V2 which introduced 
the synchronization link that the synchronous pro- 



think 



every man say 



exists arrive 
an official ^^^^ 

exists 
a Norwegian 

Figure 8: Vector derivation tree for derivation of 
(2b) 

DL is local with respect to vectors (though not with 
respect to productions, since the derivation trees of 
two synchronized UVG-DL derivations need not be 
isomorphic). The vector derivation tree can be seen 
as representing an "outline" for the derivation. Such 
a view is attractive from a linguistic perspective: if 
each vector represents a lexeme and its projection 
(where the synchronous production is the basis of 
the lexical projection that the vector represents), 
then the vector derivation tree is in fact the depen- 
dency tree of the sentence (representing direct re- 
lations between lexemes such as grammatical func- 
tion). In this respect, the vector derivation tree of 
UVG-DL is like the derivation tree of tree adjoining 



duction of vi rewrites. Unlike the standard deriva- 



tion tree for UVG-DL, the vector derivation tree 
clearly shows how the vectors (rather than the com- 
ponent rules of the vectors) were combined during 
the derivation. The vector derivation tree for the 
derivation in Figure ^ is shown in Figure ||. 



exists z, 
z a Norwegian 

exists y, 
y an official 



grammar and of D-tree grammars (DTG) (Rambow 
Vij ay- S hanker, and Weir, 1995| ), which is not sur- 



for all X, 
X a man 



think 



T 

I 



say 



T 

y 



T 

I 



Figure 7: Derivation of (2b) in a UVG-DL 

It should be clear that the vector derivation trees 
for two synchronized derivations are isomorphic, re- 
flecting the fact that our definition of SynchUVG- 



prismg, since all three formalisms share the same 
extended domain of locality. Furthermore, the vec- 
tor derivation tree of SynchUVG-DL shares with 
the the derivation tree of DTG the property that 
it refiects linguistic dependency uniformly; however, 
while the definition of DTG was motivated pre- 
cisely from considerations of dependency, the vector 
derivation tree is merely a by-product of our defi- 
nition of SynchUVG-DL, which was motivated from 
the desire to have a computationally tractable model 
of synchronization more powerful than SynchTAG.^ 
We briefly discuss a sample derivation. We start 
with the two start symbols, which are linked. We 
then apply an asynchronous production from the se- 
mantic grammar. In Figure |l^ (top) we see how 
the link is inherited by the heir nonterminal of the 
applied production. This step is repeated with two 
more asynchronous productions, yielding Figure |l^ 
(bottom) . We now apply productions for the bodies 
of the clauses, but stop short before the two syn- 
chronous productions for the arrive clause, yielding 
Figure |ll|. We see the asynchronous production of 
the syntactic arrive vector has not only inherited the 
link to its heir nonterminal, but has introduced a link 



We do not discuss modifiers in this paper for lack of 
space. 



{T}{ 

every man 

{ 



some official ^ exists y, F* 
y an official 



} 
} 




Figure 9: SynchUVG-DL grammar for quantifier 
scope disambiguation 




Figure 10: SynchUVG-DL derivation, steps 1 and 2 



of its own. Since the semantic end of the link has 
not been introduced yet, the hnks remains pending. 
We then finish the derivation to obtain the two trees 
in Figure ^ and Figure ^, with no synchronization or 
dominance links left. 



4 Formal results 

Theorem 1 SynchUVG-DL 
preservation property. 



has the language 



Proof (outline). Let be a SynchUVG-DL, G' 
and G" its left and right UVG-DL components, re- 
spectively. We construct a UVG-DL G generating 
the left-projection language of G^. G uses all the 




Figure 11: SynchUVG-DL derivation, step 3 

nonterminal symbols of G' and G", and some com- 
pound nonterminals of the form [yl, _B], A and B 
nonterminals of G' and G", respectively. G simu- 
lates Gs derivations by intermixing symbols of G' 
and symbols of G", and without generating any of 
the terminal symbols of G". Most important, each 
pair of linked nonterminals generated by Gg is rep- 
resented by G using a compound symbol. This en- 
forces the requirement of simultaneous application 
of synchronous productions to linked nonterminals. 

Each vector w of G is constructed from a pair of 
synchronous vectors {v',v") of Gg as follows. First, 
all instances of nonterminals in v" are replaced by e. 
Furthermore, for any instance _B of a right-hand side 
nonterminal of v" linked to a right-hand side non- 
terminal A oiv' , B is replaced by e and A by [A, B]. 
Then the two synchronous productions in v' and v" 
are composed into a single production in v, by com- 
posing the two left-hand sides in a compound symbol 
and by concatenating the two right-hand sides. Fi- 
nally, to simulate link inheritance in derivations of 
Gs, each asynchronous production in v' and v" is 
transferred to v, either without any change, or by 
composing with some nonterminal G both its left- 
hand side and the heir nonterminal in its right-hand 
side. Note that there are finitely many choices for 
the last step, and each choice gives a different vector 
in G, simulating the application of v' and v" to a set 
of (occurrences of) nonterminals in a particular link 
configuration in a sentential form of Gs- ■ 

We now introduce a representation for sets of 
derivation trees in a UVG-DL G. A parse tree in 
G is an ordered tree representing a derivation in G 
and encoding at each node the production p used to 
start the corresponding subderivation and the mul- 
tiset of productions / used in that subderivation. A 



parse forest in G is a directed acyclic graph which 
is ordered and bipartite. (We use ideas originally 
developed in (Lang, 1991) for the context-free case.) 
Nodes of the graph are of two different types, called 
and-nodes and or-nodes, respectively, and each di- 
rected arc connects nodes of different types. A parse 
forest in G represents a set T of parse trees in G if 
the following holds. When starting at a root node 
and walking through the graph, if we follow exactly 
one of the outgoing arcs at each or-node, and all of 
the outgoing arcs at each and-node, we obtain a tree 
in T modulo the removal of the or-nodes. Further- 
more, every tree in T can be obtained in this way. 

Lemma 2 Let G be a UVG-DL and let q > 1 be 
a natural number. The parse forest representing the 
set of all parse trees in G with no more than q vectors 
can be constructed in an amount of time bounded by 
a polynomial function of q. ■ 

Let Gs be a SynchUVG-DL, G" and G" its left 
and right UVG-DL components, respectively. For 
a parse tree r in G', we denote as T(r) the set 
of all parse trees in G" that are synchronous with 
T according to Gg. The parse-to-forest translation 
problem for Gg takes as input a parse tree r in G' 
and gives as output a parse forest representation for 
T(r). If Gs is lexicalized, such a parse forest has size 
bounded by a polynomial function of | t | , despite the 
fact that the size of T{t) can be exponentially larger 
than the size of r. In fact, we have a stronger result. 

Theorem 3 The parse-to-forest translation prob- 
lem for a lexicalized SynchUVG-DL can be computed 
in polynomial time. 

Proof (outline). Let Gs be a SynchUVG-DL 
with G' and G" its left and right UVG-DL com- 
ponents, respectively. Let t be a parse tree in G' 
and TT be the parse forest representing T{t). The 
construction of tt consists of two stages. 

In the first stage, we construct the vector deriva- 
tion tree 7 associated with r. Let q be the number 
of nodes of 7. We also construct a parse forest tt^ 
representing the set of all parse trees in G" with no 
more than q vectors. This stage takes polynomial 
time in the size of r, since 7 can be constructed 
from r in linear time and tt^ can be constructed as 
in Lemma ^. 

In the second stage, we remove from tt^ all the 
parse trees not in tt. This completes the construc- 
tion, since the set of parse trees represented by tt is 
included in the set of parse trees represented by tt^. 
Let Ur and F be the root node and the set of all nodes 
of 7, respectively. For n G F, out{n) denotes the set 
of all children of n. We call family the set {n^} and 
any nonempty subset of out{n), n G F. The main 



idea is to associate a set of families J^n to each node 
n of TTq, such that the following condition is satis- 
fied. A family F belongs to J-^ if and only if at least 
one subderivation in G" represented at n induces a 
forest of vector derivation trees whose root nodes 
are all and only the nodes in F. Each Tn can eas- 
ily be computed visiting tt^ in a bottom-up fashion. 
Crucially, we "block" a node of iTq if we fail in the 
construction of Tn. We claim that each set IFn has 
size bounded by the number of nodes in 7. This can 
be shown using the fact that all derivation trees rep- 
resented at a node of -Kq employ the same multiset of 
productions of G". From the above claim, it follows 
that TTq can be processed in time polynomial in the 
size of T. Finally, we obtain tt simply by removing 
from TTg all nodes that have been blocked. ■ 

5 Conclusion 

We have presented SynchUVG-DL, a synchronous 
system which has restricted formal power, is com- 
putationally tractable, and which handles the 
quantifier-raising data. In addition, SynchUVG-DL 
can be used for modeling the syntax of languages 
with syntactic constructions which have been ar- 
gued to be beyond the formal power of TAG, such 
as scrambling in German and many other lan- 
guages ( Rambow, 1994|) or w/t- movement in Kash - 



miri (Rambow, Vijay-Shanker, and Weir, 1995). 
SynchUVG-DL can be used to synchronize a syn- 
tactic grammar for these languages either with a se- 
mantic grammar, or with the syntactic grammar of 
another language for machine translation applica- 
tions. However, SynchUVG-DL cannot handle the 
list of cases listed in ( ^hieber, 1994 ). These pose a 
problem for SynchUVG-DL for the same reason that 
they pose a problem for other local synchronous sys- 
tems: the (syntactic) dependency structures repre- 
sented by the two derivations are different. These 
cases remain an open research issue. 
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